Supplemental Material for Parallel Sampling of DP Mixture Models using Sub-Clusters Splits
نویسندگان
چکیده
In this section, we show the derivation of the posterior distribution over cluster-weights, π, conditioned on the cluster labels, z. We begin with the definition of a Dirichlet process from [1]. Definition A.1 (Dirichlet Process). Let H be a measure on a measureable space, Ω. If for any finite partition, (A1, A2, · · · , AK) of the space, the measure, G, on the partition follows the following Dirichlet distribution (G(A1), G(A2), · · · , G(AK)) ∼ Dir(αH(A1), αH(A2), · · · , αH(AK)), (1) for some positive scalar α, then G is said to be a Dirichlet process with concentration parameter α and base measure H .
منابع مشابه
Parallel Sampling of DP Mixture Models using Sub-Clusters Splits
We present an MCMC sampler for Dirichlet process mixture models that can be parallelized to achieve significant computational gains. We combine a nonergodic, restricted Gibbs iteration with split/merge proposals in a manner that produces an ergodic Markov chain. Each cluster is augmented with two subclusters to construct likely split moves. Unlike some previous parallel samplers, the proposed s...
متن کاملParallel Sampling of DP Mixture Models using Sub-Cluster Splits
We present an MCMC sampler for Dirichlet process mixture models that can be parallelized to achieve significant computational gains. We combine a nonergodic, restricted Gibbs iteration with split/merge proposals in a manner that produces an ergodic Markov chain. Each cluster is augmented with two subclusters to construct likely split moves. Unlike some previous parallel samplers, the proposed s...
متن کاملSampling in computer vision and Bayesian nonparametric mixtures
The field of computer vision focuses on understanding and reasoning about the visual world. Due to the complexity of this problem, researchers often focus on one specific component of this large task, such as segmentation or recognition. This modularized approach necessitates the combination of each separate component, which Bayesian formulations handle in a mathematically consistent framework....
متن کاملSupplemental Material for Parallel Sampling of HDPs using Sub-Cluster Splits
In the following supplemental material we provide some additional details and derivations for the paper. We begin by showing how to calculate the joint distribution of p(β, z), marginalizing out π, in Section 1. Then, in Section 2 we consider looking at joint log-likelihoods of HDP topic models and show that the typical set of the distribution is very far from the mode. In Sections 3-4, we give...
متن کاملParallel Sampling of HDPs using Sub-Cluster Splits
We develop a sampling technique for Hierarchical Dirichlet process models. The parallel algorithm builds upon [1] by proposing large split and merge moves based on learned sub-clusters. The additional global split and merge moves drastically improve convergence in the experimental results. Furthermore, we discover that cross-validation techniques do not adequately determine convergence, and tha...
متن کامل